Machine vision inspection system and method for obtaining an image with an extended depth of field

ABSTRACT

A method for providing an extended depth of field (EDOF) image includes: Periodically modulating an imaging system focus position at a high frequency; using an image exposure comprising discrete image exposure increments acquired at discrete focus positions during an image integration time comprising a plurality of modulation periods of the focus position; and using strobe operations having controlled timings configured to define a set of evenly spaced focus positions for the image exposure increments. The timings are configured so that adjacent focus positions in the set are acquired at times that are separated by at least one reversal of the direction of change of the focus position during its periodic modulation. This solves practical timing problems that may otherwise prevent obtaining closely spaced discrete image exposure increments during high frequency focus modulation. Deconvolution operations may be used to improve clarity in the resulting EDOF image.

FIELD

The invention relates generally to machine vision inspection systems,and more particularly to extended depth-of-field imaging operations.

BACKGROUND

Precision machine vision inspection systems (or “vision systems” forshort) are used to obtain precise dimensional measurements of objectsand to inspect various other object characteristics. Such systems mayinclude a computer, a camera and optical system, and a precision stagethat moves to allow workpiece traversal and inspection. One exemplaryprior art system, characterized as a general-purpose “off-line”precision vision system, is the QUICK VISION® series of PC-based visionsystems and QVPAK® software available from Mitutoyo America Corporation(MAC), located in Aurora, Ill. The features and operation of the QUICKVISION® series of vision systems and the QVPAK® software are generallydescribed, for example, in the QVPAK 3D CNC Vision Measuring MachineUser's Guide, published January 2003, and the QVPAK 3D CNC VisionMeasuring Machine Operation Guide, published September 1996, each ofwhich is hereby incorporated by reference in their entirety. This typeof system uses a microscope-type optical system and moves the stage soas to provide inspection images of either small or relatively largeworkpieces at various magnifications.

General-purpose precision machine vision inspection systems aregenerally programmable to provide automated video inspection. Suchsystems typically include GUI features and predefined image analysis“video tools” such that operation and programming can be performed by“non-expert” operators. For example, U.S. Pat. No. 6,542,180, which isincorporated herein by reference in its entirety, teaches a visionsystem that uses automated video inspection including the use of variousvideo tools.

The machine control instructions including the specific inspection eventsequence (i.e., how to acquire each image and how to analyze/inspecteach acquired image) are generally stored as a “part program” or“workpiece program” that is specific to the particular workpiececonfiguration. For example, a part program defines how to acquire eachimage, such as how to position the camera relative to the workpiece, atwhat lighting level, at what magnification level, etc. Further, the partprogram defines how to analyze/inspect an acquired image, for example,by using one or more video tools such as autofocus video tools.

Video tools (or “tools” for short) and other graphical user interfacefeatures may be used manually to accomplish manual inspection and/ormachine control operations (in “manual mode”). Their set-up parametersand operation can also be recorded during learn mode, in order to createautomatic inspection programs, or “part programs.” Video tools mayinclude, for example, edge-/boundary-detection tools, autofocus tools,shape- or pattern-matching tools, dimension-measuring tools, and thelike.

In some applications, it is desirable to operate an imaging system of amachine vision inspection system to collect an image with an extendeddepth of field (EDOF), such that the depth of field is larger than thatprovided by the optical imaging system at a single focus position.Various methods are known for collecting an image with an extended depthof field. One such method is to collect an image “stack,” consisting ofa plurality of congruent or aligned images focused at differentdistances throughout a focus range. A mosaic image of the field of viewis constructed from the image stack, wherein each portion of the fieldof view is extracted from the particular image that shows that portionwith the best focus. However, this method is relatively slow. As anotherexample, Nagahara et al. (“Flexible Depth of Field Photography.”Proceedings of the European Conference on Computer Vision, October 2008)discloses a method wherein a single image is exposed along a pluralityof focus distances during its exposure time. This image is relativelyblurry, but contains image information acquired over the plurality offocus distances. It is deconvolved using a known or predetermined blurkernel to obtain a relatively clear image with an extended depth offield. In the method described in Nagahara, the focal distance isaltered by translating the image detector along an optical axis of animaging system. As a result, different focal planes are focused on thedetector at different times during exposure. However, such a method isrelatively slow and mechanically complex. Furthermore, altering thedetector position may have detrimental effects on repeatability and/oraccuracy when it is used for acquiring fixed focus inspection images,which must be used for precision measurements (e.g., for accuracies onthe order of a few micrometers) and the like. An improved method forproviding an extended depth of field (EDOF) image is desirable, whichmay be performed at high speed without relying on mechanical translationof optical components.

SUMMARY

A typical high speed variable focus lens may modulate a focus positionin a sinusoidal manner (as opposed to linear), which will generally notprovide an even or “balanced” exposure throughout an entire cycle offocus position modulation which may be used to acquire an extended depthof field (EDOF) image. In contrast, in various applications using a highspeed variable focus lens it is desirable to provide an even or“balanced” exposure throughout an entire cycle of focus positionmodulation which may be used to acquire an extended depth of field(EDOF) image.

A method is disclosed for operating an imaging system of a machinevision inspection system in order to provide at least one EDOF imagethat has a larger depth of field than the imaging system in a singlefocal position. In various implementations, the method includes exposinga preliminary EDOF image using an image exposure comprising a pluralityof discrete image exposure increments, according to principles disclosedherein.

In various implementations, the method may include placing a workpiecein a field of view of the machine vision inspection system. A focusposition of variable focus imaging system is periodically modulated,preferably without macroscopically adjusting the spacing betweenelements in the imaging system, wherein the focus position isperiodically modulated over a plurality of focus positions along a focusaxis direction in a focus range including a surface height of theworkpiece, at a modulation frequency of at least 3 kHz.

In various implementations, a preliminary EDOF image is exposed using animage exposure comprising a plurality of discrete image exposureincrements acquired at respective discrete focus positions during animage integration time comprising a plurality of periods of theperiodically modulated focus position, wherein:

-   -   the plurality of discrete image exposure increments are each        determined by a respective instance of an illumination source        strobe operation, or a camera shutter strobe operation, that has        a respective controlled timing that defines the discrete focus        position of the corresponding discrete image exposure increment;    -   the respective controlled timings are distributed over the        plurality of periods of the periodically modulated focus        position, and are configured to provide a set of discrete focus        positions which are approximately evenly spaced along the focus        axis direction; and    -   the respective controlled timings are furthermore configured so        that for a plurality of adjacent pairs of discrete focus        positions in the set, when a first controlled timing provides a        first discrete focus position set of the adjacent pair, a second        controlled timing that provides a second discrete focus position        of the adjacent pair is controlled to have a delay relative to        the first controlled timing such that the second controlled        timing is controlled to occur after N reversals of the direction        of change of the focus position during its periodic modulation        following the first controlled timing, where N is at least 1.

In various implementations, the preliminary EDOF image may be processedto remove blurred image contributions occurring in the focus rangeduring the image integration time to provide an extended depth of field(EDOF) image that is substantially focused throughout a larger depth offield than the imaging system provides at a single focal position. Insome implementations, such processing may include performingdeconvolution operations using a blur kernel that characterizes theimaging system throughout its focus range (e.g., an integrated pointspread function).

In some implementations, each discrete image exposure incrementcomprises a combination of an increment exposure duration and anillumination intensity used during the increment exposure duration suchthat each discrete image exposure increment provides nominally equalexposure energy to the preliminary EDOF image. In some implementations,the increment exposure durations corresponding to different focuspositions are adjusted such that approximately the same amount of focusposition change occurs during each of the discrete image exposureincrements.

In some implementations disclosed herein a continuous (includingpartially continuous) EDOF image exposure may be used. However, onedrawback of such methods may be that the associated EDOF image exposuremay not be uniform throughout the focus range, which is detrimental in anumber of implementations. An alternative method emphasized above inthis summary includes using a plurality of discrete image exposureincrements to acquire a preliminary EDOF image in a focus range of afast variable focus lens (or imaging system), as outlined above. Such amethod of EDOF image exposure may be more desirable in that it may be amore adaptable, accurate, and/or robust method in variousimplementations.

It should be appreciated that when using such a method in conjunctionwith a very high speed periodically modulated variable focus lens (e.g.,a TAG lens), that the focus position may change so quickly (as aninherent feature of the variable focus lens) that significant timing,control, and “exposure amount” problems may arise in practical systems.In order to provide a practical solution to such problems, in variousimplementations disclosed herein, the discrete image exposure incrementsthat are used as constituents of an EDOF image exposure are acquiredover a plurality of the periodic focus modulations, using a controlledtiming configured according to certain principles outlined above, anddisclosed in greater detail and variety below.

BRIEF DESCRIPTION OF THE DRAWINGS

The foregoing aspects and many of the attendant advantages of thisinvention will become more readily appreciated as the same become betterunderstood by reference to the following detailed description, whentaken in conjunction with the accompanying drawings, wherein:

FIG. 1 is a diagram showing various typical components of ageneral-purpose precision machine vision inspection system;

FIG. 2 is a block diagram of a control system portion and a visioncomponents portion of a machine vision inspection system similar to thatof FIG. 1 and including features disclosed herein;

FIG. 3 shows a schematic diagram of a first embodiment of an EDOFimaging system that may be adapted to a machine vision inspection systemand operated according to the principles disclosed herein;

FIG. 4 shows an exemplary timing diagram for a focal height during animage exposure as may be used in one embodiment of an EDOF imagingsystem (e.g., the imaging system of FIG. 3) according to principlesdisclosed herein;

FIG. 5 shows a schematic diagram of a second embodiment of an EDOFimaging system that may be adapted to a machine vision inspection systemand operated according to the principles disclosed herein;

FIG. 6A is a graph characterizing a first embodiment of an opticalfilter which may be used at a Fourier plane of an imaging system, inorder to perform optical deconvolution of an image from an EDOF imagingsystem and provide a relatively clear EDOF image in real time;

FIG. 6B is a graph characterizing a second embodiment of an opticalfilter which may be used at a Fourier plane of an imaging system;

FIG. 7 is a flow diagram showing one embodiment of a method foroperating an imaging system of a machine vision inspection system inorder to perform computational deconvolution of a preliminary image froman EDOF imaging system and provide a relatively clear EDOF imageapproximately in real time;

FIGS. 8A-8C show exemplary timing diagrams illustrating various aspectsof three different image exposure implementations suitable for an EDOFimaging system (e.g., the imaging system of FIG. 3), including the useof discrete image exposure increments, according to principles disclosedherein.

FIG. 9 shows a timing diagram including certain details of one exemplaryimplementation of a controlled timing configuration that may be used todefine a discrete focus position and certain other characteristicsassociated with corresponding discrete image exposure increment.

FIG. 10 is a flow diagram showing one embodiment of a method foroperating an imaging system (e.g., in an inspection system) in order toprovide at least one EDOF image that has a larger depth of field thanthe imaging system in a single focal position, and including the use ofdiscrete image exposure increments, according to principles disclosedherein.

DETAILED DESCRIPTION

FIG. 1 is a block diagram of one exemplary machine vision inspectionsystem 10 usable in accordance with methods described herein. Themachine vision inspection system 10 includes a vision measuring machine12 that is operably connected to exchange data and control signals witha controlling computer system 14. The controlling computer system 14 isfurther operably connected to exchange data and control signals with amonitor or display 16, a printer 18, a joystick 22, a keyboard 24, and amouse 26. The monitor or display 16 may display a user interfacesuitable for controlling and/or programming the operations of themachine vision inspection system 10. It will be appreciated that invarious embodiments, a touchscreen tablet or the like may be substitutedfor and/or redundantly provide the functions of any or all of thecomputer system 14, the display 16, the joystick 22, the keyboard 24,and the mouse 26.

Those skilled in the art will appreciate that the controlling computersystem 14 may generally consist of any computing system or device.Suitable computing systems or devices may include personal computers,server computers, minicomputers, mainframe computers, distributedcomputing environments that include any of the foregoing, and the like.Such computing systems or devices may include one or more processorsthat execute software to perform the functions described herein.Processors include programmable general-purpose or special-purposemicroprocessors, programmable controllers, application-specificintegrated circuits (ASICs), programmable logic devices (PLDs), or thelike, or a combination of such devices. Software may be stored inmemory, such as random-access memory (RAM), read-only memory (ROM),flash memory, or the like, or a combination of such components. Softwaremay also be stored in one or more storage devices, such as optical-baseddisks, flash memory devices, or any other type of non-volatile storagemedium for storing data. Software may include one or more programmodules that include routines, programs, objects, components, datastructures, and so on that perform particular tasks or implementparticular abstract data types. In distributed computing environments,the functionality of the program modules may be combined or distributedacross multiple computing systems or devices and accessed via servicecalls, either in a wired or wireless configuration.

The vision measuring machine 12 includes a moveable workpiece stage 32and an optical imaging system 34 that may include a zoom lens orinterchangeable lenses. The zoom lens or interchangeable lensesgenerally provide various magnifications for the images provided by theoptical imaging system 34. The machine vision inspection system 10 isalso described in commonly assigned U.S. Pat. Nos. 7,454,053; 7,324,682;8,111,905; and 8,111,938, each of which is incorporated herein byreference in its entirety.

FIG. 2 is a block diagram of a control system portion 120 and a visioncomponents portion 200 of a machine vision inspection system 100 similarto the machine vision inspection system of FIG. 1, and includingfeatures as described herein. As will be described in more detail below,the control system portion 120 is utilized to control the visioncomponents portion 200. The vision components portion 200 includes anoptical assembly portion 205, light sources 220, 230, and 240, and aworkpiece stage 210 having a central transparent portion 212. Theworkpiece stage 210 is controllably movable along X and Y axes that liein a plane that is generally parallel to the surface of the stage wherea workpiece 20 may be positioned. The optical assembly portion 205includes a camera system 260, an interchangeable objective lens 250, andmay include a turret lens assembly 280 having lenses 286 and 288.Alternatively to the turret lens assembly, a fixed or manuallyinterchangeable magnification-altering lens, or a zoom lensconfiguration, or the like, may be included.

The optical assembly portion 205 is controllably movable along a Z axisthat is generally orthogonal to the X and Y axes by using a controllablemotor 294 that drives an actuator to move the optical assembly portion205 along the Z axis to change the focus of the image of the workpiece20. The controllable motor 294 is connected to an input/output interface130 via a signal line 296.

A workpiece 20, or a tray or fixture holding a plurality of workpieces20, which is to be imaged using the machine vision inspection system100, is placed on the workpiece stage 210. The workpiece stage 210 maybe controlled to move relative to the optical assembly portion 205, suchthat the interchangeable objective lens 250 moves between locations on aworkpiece 20, and/or among a plurality of workpieces 20. One or more ofa stage light 220, a coaxial light 230, and a surface light 240 (e.g., aring light) (collectively light sources) may emit source light 222, 232,and/or 242, respectively, to illuminate the workpiece or workpieces 20.The light source 230 may emit light 232 along a path including a mirror290. The source light is reflected or transmitted as workpiece light255, and the workpiece light used for imaging passes through theinterchangeable objective lens 250 and the turret lens assembly 280 andis gathered by the camera system 260. The image of the workpiece(s) 20,captured by the camera system 260, is output on a signal line 262 to thecontrol system portion 120. The light sources 220, 230, and 240 may beconnected to the control system portion 120 through signal lines orbusses 221, 231, and 241, respectively. To alter the imagemagnification, the control system portion 120 may rotate the turret lensassembly 280 along axis 284 to select a turret lens through a signalline or bus 281.

As shown in FIG. 2, in various exemplary embodiments, the control systemportion 120 includes a controller 125, the input/output interface 130, amemory 140, a workpiece program generator and executor 170, and a powersupply portion 190. Each of these components, as well as the additionalcomponents described below, may be interconnected by one or moredata/control busses and/or application programming interfaces, or bydirect connections between the various elements.

The input/output interface 130 includes an imaging control interface131, a motion control interface 132, a lighting control interface 133,and a lens control interface 134. The imaging control interface 131 mayinclude an extended depth of field (EDOF) mode 131 e, which a user mayselect to collect at least one image of a workpiece with a depth offield that is greater than what may be provided by the vision componentsportion 200 when focused at a single focus position. The lens controlinterface 134 may comprise an EDOF lens controller including a lensfocus driving circuit and/or routine, or the like. The operations andcomponents associated with an extended depth of field mode and an EDOFlens control interface and/or controller are described further belowwith reference to FIGS. 3-7. The motion control interface 132 mayinclude a position control element 132 a, and a speed/accelerationcontrol element 132 b although such elements may be merged and/orindistinguishable. The lighting control interface 133 includes lightingcontrol elements 133 a, 133 n, and 133 fl that control, for example, theselection, power, on/off switch, and strobe pulse timing, if applicable,for the various corresponding light sources of the machine visioninspection system 100.

The memory 140 may include an image file memory portion 141, anedge-detection memory portion 140 ed, a workpiece program memory portion142 that may include one or more part programs, or the like, and a videotool portion 143. The video tool portion 143 includes video tool portion143 a and other video tool portions (e.g., 143 n) that determine theGUI, image-processing operation, etc., for each of the correspondingvideo tools, and a region of interest (ROI) generator 143 roi thatsupports automatic, semi-automatic, and/or manual operations that definevarious ROIs that are operable in various video tools included in thevideo tool portion 143. The video tool portion also includes anautofocus video tool 143 af that determines the GUI, image-processingoperation, etc., for focus height measurement operations. In the contextof this disclosure, and as known by one of ordinary skill in the art,the term “video tool” generally refers to a relatively complex set ofautomatic or programmed operations that a machine vision user canimplement through a relatively simple user interface (e.g., a graphicaluser interface, editable parameter windows, menus, and the like),without creating the step-by-step sequence of operations included in thevideo tool or resorting to a generalized text-based programminglanguage, or the like. For example, a video tool may include a complexpre-programmed set of image-processing operations and computations thatare applied and customized in a particular instance by adjusting a fewvariables or parameters that govern the operations and computations. Inaddition to the underlying operations and computations, the video toolcomprises the user interface that allows the user to adjust thoseparameters for a particular instance of the video tool. For example,many machine vision video tools allow a user to configure a graphicalregion of interest (ROI) indicator through simple “handle dragging”operations using a mouse, in order to define the location parameters ofa subset of an image that is to be analyzed by the image-processingoperations of a particular instance of a video tool. It should be notedthat the visible user interface features are sometimes referred to asthe video tool with the underlying operations being included implicitly.

The signal lines or busses 221, 231, and 241 of the stage light 220, thecoaxial lights 230 and 230′, and the surface light 240, respectively,are all connected to the input/output interface 130. The signal line 262from the camera system 260 and the signal line 296 from the controllablemotor 294 are connected to the input/output interface 130. In additionto carrying image data, the signal line 262 may carry a signal from thecontroller 125 that initiates image acquisition.

One or more display devices 136 (e.g., the display 16 of FIG. 1) and oneor more input devices 138 (e.g., the joystick 22, keyboard 24, and mouse26 of FIG. 1) can also be connected to the input/output interface 130.The display devices 136 and input devices 138 can be used to display auser interface that may include various graphical user interface (GUI)features that are usable to perform inspection operations, and/or tocreate and/or modify part programs, to view the images captured by thecamera system 260, and/or to directly control the vision componentsportion 200. The display devices 136 may display user interface featuresassociated with the autofocus video tool 143 af.

In various exemplary embodiments, when a user utilizes the machinevision inspection system 100 to create a part program for the workpiece20, the user generates part program instructions by operating themachine vision inspection system 100 in a learn mode to provide adesired image-acquisition training sequence. For example, a trainingsequence may comprise positioning a particular workpiece feature of arepresentative workpiece in the field of view (FOV), setting lightlevels, focusing or autofocusing, acquiring an image, and providing aninspection training sequence applied to the image (e.g., using aninstance of one of the video tools on that workpiece feature). The learnmode operates such that the sequence(s) are captured or recorded andconverted to corresponding part program instructions. Theseinstructions, when the part program is executed, will cause the machinevision inspection system to reproduce the trained image acquisition andcause inspection operations to automatically inspect that particularworkpiece feature (that is the corresponding feature in thecorresponding location) on a run mode workpiece, or workpieces, whichmatches the representative workpiece used when creating the partprogram. The systems and methods disclosed herein are particularlyuseful during such learn mode and/or manual operations, in that a usermay see an EDOF video image in real time while navigating a workpiecefor visual inspection and/or workpiece program creation. The user neednot continually refocus high-magnification images depending on theheight of various microscopic features on the workpiece, which can betedious and time-consuming, especially at high magnifications.

FIG. 3 shows a schematic diagram of a first embodiment of an EDOFimaging system 300 that may be adapted to a machine vision inspectionsystem and operated according to the principles disclosed herein. Theimaging system 300 is configurable to provide at least one image of aworkpiece that has a larger depth of field than the imaging system in asingle focal position (e.g., 10-20 times larger, or more, in variousembodiments). The imaging system 300 comprises a light source 330 thatis configurable to illuminate a workpiece in a field of view of theimaging system 300, an objective lens 350, a relay lens 351, a relaylens 352, a variable focal length lens 370, a tube lens 386, and acamera system 360.

In operation, the light source 330 is configurable to emit source light332 along a path including a mirror 390 to a surface of a workpiece 320,the objective lens 350 receives workpiece light 332 including workpiecelight that is focused at a focus position FP proximate to the workpiece320, and outputs the workpiece light 355 to the relay lens 351. Therelay lens 351 receives the workpiece light 355 and outputs it to therelay lens 352. The relay lens 352 receives the workpiece light 355 andoutputs it to the variable focal length lens 370. Together, the relaylens 351 and the relay lens 352 provide a 4 f optical relay between theobjective lens 350 and the variable focal length lens 370 in order toprovide constant magnification for each Z height of the workpiece 320and/or focus position FP. The variable focal length lens 370 receivesthe workpiece light 355 and outputs it to the tube lens 386. Thevariable focal length lens 370 is electronically controllable to varythe focus position FP of the imaging system during one or more imageexposures. The focus position FP may be moved within a range R bound bya focus position FP1 and a focus position FP2. It should be appreciatedthat in some embodiments, the range R may be selected by a user, e.g.,in the EDOF mode 131 e of the imaging control interface 131.

In various embodiments, a machine vision inspection system comprises acontrol system (e.g., the control system portion 120) that isconfigurable to control the variable focal length lens 370 toperiodically modulate a focus position of the imaging system 300. Insome embodiments, the variable focal length lens 370 may very rapidlyadjust or modulate the focus position (e.g., periodically, at a rate ofat least 300 Hz, or 3 kHz, or much higher). In some embodiments, therange R may be as large as 10 mm (for a 1X objective lens 350). Invarious embodiments, the variable focal length lens 370 isadvantageously chosen such that it does not require any macroscopicmechanical adjustments imaging system and/or adjustment of the distancebetween the objective lens 350 and the workpiece 320 in order to changethe focus position FP. In such case, the EDOF image may be provided at ahigh rate, and furthermore there are no macroscopic adjustment elementsnor associated positioning non-repeatability to degrade accuracy whenthe same imaging system is used for acquiring fixed focus inspectionimages, which must be used for precision measurements (e.g., foraccuracies on the order of a few micrometers) and the like. For example,in some embodiments it is desirable to use the EDOF image as a displayimage for a user, and later terminate the periodic modulating of thefocus position (e.g., using the previously described EDOF mode controlelement 131 e, or automatic termination based on an active measurementoperation, or the like) to provide a fixed focus position for theimaging system. Then the system may be used to expose a measurementimage of a particular feature using the imaging system with the fixedfocus position; and that stable high-resolution measurement image may beprocessed to provide an accurate measurement of the workpiece.

In some embodiments, the variable focal length lens 370 is a tunableacoustic gradient index of refraction lens. A tunable acoustic gradientindex of refraction lens is a high-speed variable focal length lens thatuses sound waves in a fluid medium to modulate a focus position and mayperiodically sweep a range of focal lengths at a frequency of severalhundred kHz. Such a lens may be understood by the teachings of thearticle, “High-speed varifocal imaging with a tunable acoustic gradientindex of refraction lens” (Optics Letters, Vol. 33, No. 18, Sep. 15,2008), which is hereby incorporated by reference in its entirety.Tunable acoustic gradient index lenses and related controllable signalgenerators are available, for example, from TAG Optics, Inc., ofPrinceton, N.J. The SR38 series lenses, for example, are capable ofmodulation up to 1.0 MHz.

The variable focal length lens 370 may be driven by an EDOF lenscontroller 374, which may generate a signal to control the variablefocal length lens 370. In one embodiment, the EDOF lens controller 374may be a commercial controllable signal generators such as that referredto above. In some embodiments, the EDOF lens controller 374 may beconfigured or controlled by a user and/or an operating program throughthe imaging control interface 131 and/or a user interface of the EDOFmode 131 e and/or the lens control interface 134, outlined previouslywith reference to FIG. 2. In some embodiments, the variable focal lengthlens 370 may be driven using a periodic signal such that the focusposition FP is modulated sinusoidally over time, at a high frequency.For example, in some exemplary embodiments, a tunable acoustic gradientindex of refraction lens may be configured for focal scanning rates ashigh as 400 kHz, although it should be appreciated that slower focusposition adjustments and/or modulation frequencies may be desirable invarious embodiments and/or applications. For example, in variousembodiments a periodic modulation of 300 Hz, or 3 kHz, or the like maybe used. In embodiments where such slower focus position adjustments areused, the variable focal length lens 370 may comprise controllable fluidlens, or the like.

The embodiment of an EDOF imaging system shown in FIG. 3 is usable whenan EDOF imaging system and associated signal processing is configured toperform computational deconvolution of a preliminary image from an EDOFimaging system and provide a relatively clear EDOF image approximatelyin real time. For example, a control system (e.g., the control systemportion 120 shown in FIG. 2) is configured to collect a firstpreliminary image during the course of at least one sweep of themodulated focus position throughout an EDOF focus range during the imageexposure, and process the first preliminary image, which may be blurry,to determine a relatively clear image. In one embodiment, thepreliminary image may be processed or deconvolved using a known orpredetermined point spread function (PSF) corresponding to the focusrange of the preliminary image. A point spread function P(FP)characterizes a blur circle, i.e., a circular image of a point lightsource at a given distance from an imaging system as a function of aradius r of the blur circle and the focus position FP. A point spreadfunction may be determined experimentally for an imaging system (e.g.,the imaging system 300) or it may be estimated using point spreadfunctions modeled on functions such as a pill box or a Gaussian curve,or using basic diffraction principles, e.g., Fourier optics, accordingto known methods. Such point spread functions at various focus distanceswithin a focus range may be weighted according to their expectedexposure contributions or applicability. For example, when the focusdistance moves during an exposure, each focus distance will contributeto an image exposure for a corresponding time period within thatexposure, and a point spread function corresponding to that distance maybe weighted accordingly. Such weighted point spread functioncontributions may be summed or integrated over an expected focus rangeR. Alternatively, when the focus distance change is a known function oftime, such point spread function contributions may be integrated over aperiod of time corresponding to a sweep of the expected focus range R,analogous to the approach indicated with reference to EQUATION 3 below.

For an imaging system with a modulated focus position, an integratedpoint spread function h which follows the relation:

h=∫ ₀ ^(T) P(FP(t))dt  Eq. 1

where P(FP(t)) is a point spread function and FP(t) is thetime-dependent focal position. A focus position of an imaging system ofa machine vision inspection system may be modulated as a function oftime t, over a total integration time T, corresponding to an imageexposure or integration time of the first preliminary image.

Deconvolution of the first preliminary image may be understood as aninverse operation that deconvolves a high depth of field image exposedover a range of focus positions having respective durations in theexposure, from an integrated point spread function h, which in someapplications may be referred to as a “blur function.” The firstpreliminary image may be represented as a two-dimensional functiong(x,y) which is a convolution of an extended depth of field image f(x,y)(corresponding to an image array with dimensions m×n) with theintegrated point spread function h by the equation:

g(x,y)=f*h=Σ _(m)Σ_(n) f(m,n)h(x−m,y−n)  Eq. 2

In the frequency domain, this convolution may be represented by theproduct of the Fourier transforms off and h, represented as F and H:

G=F·H  Eq. 3

The Fourier transforms of f and h may be determined efficiently using afast Fourier transform (FFT) algorithm. The EDOF image (in the frequencydomain) may be determined by processing the image G (i.e., multiplyingit) by an inverse of H denoted here as H_(r). The inverse H_(r) may becomputed by several known methods. For example, a simple pseudo inverseof H may be determined by the equation:

$\begin{matrix}{H_{r} = \frac{H^{*}}{{H}^{2} + k}} & {{Eq}.\mspace{14mu} 4}\end{matrix}$

where H* is the complex conjugate of the H, and k is a real numberchosen empirically based on characteristics of the imaging system 300.In one exemplary embodiment, k is 0.0001. Finally, the extended depth offield image f may be computed as:

$\begin{matrix}{{f( {x,y} )} = {{g*h_{r}} = {{\mathcal{F}^{- 1}( {G \cdot H_{r}} )} = {\mathcal{F}^{- 1}\{ {G \cdot \frac{H^{*}}{{H}^{2} + k}} \}}}}} & {{Eq}.\mspace{14mu} 5}\end{matrix}$

A more robust alternative to the pseudo inverse may be computedaccording to a Wiener Deconvolution or a Lucy-Richardson iterativealgorithm, which are described in Digital Image Processing by Kenneth R.Castleman (Prentice-Hall, Inc., 1996). Additionally, processing theimage may comprise block-based denoising.

In a different embodiment, as described in greater detail below withrespect to FIGS. 5 and 6, a deconvolution may be performed opticallyusing a passive optical filter placed in a Fourier plane of an EDOFimaging system according to basic methods of Fourier optics, in order toprovide a relatively clear EDOF image in real time.

In exemplary embodiments, the imaging system 300 may provide a firstpreliminary image, which is a blurred image including informationacquired throughout a desired focus range during its exposure. The firstpreliminary image may then be computationally processed as outlinedabove to provide an extended depth of field image that comprises a depthof field that is larger than the imaging system 300 may provide at asingle focal position (e.g., 100 times larger). For example, at a singlefocal position, the depth of field may be 90 μm and an extended depth offield image provided using the same embodiment of the imaging system 300may be as large as 9 mm.

FIG. 4 shows an exemplary timing diagram 400 for a focal height duringan image exposure as may be used in one embodiment of an EDOF imagingsystem (e.g., the imaging system 300) according to principles disclosedherein. The timing diagram 400 additionally shows exposure times of acamera of the imaging system. Generally speaking, EDOF image exposures,also referred to as frame exposures in the following description, may beperformed by the imaging system over at least one sweep of themodulation of the focal height of the imaging system over a desiredfocus range during the exposure. In the particular example shown in thetiming diagram 400, a frame exposure is performed corresponding to atleast one cycle of a periodic modulation of the focal height of theimaging system over a desired focus range. High-speed periodicmodulation is conveniently performed using a tunable acoustic gradientindex of refraction lens. More specifically, in one embodiment, thefollowing steps reflected in FIG. 4 are repeated at least one time toprovide an EDOF image that is substantially focused throughout a largerdepth of field than the imaging system provides at a single focalposition:

-   -   periodically modulating a focus position (focal plane) of the        imaging system over a plurality of focus positions along a focus        axis direction without macroscopically adjusting the spacing        between elements in the imaging system, the focus position        periodically modulated in a focus range including a surface        height of the workpiece at a frequency of at least 300 Hz;    -   exposing a first preliminary image during an image integration        time while modulating the focus position in the focus range; and    -   processing the first preliminary image to remove blurred image        contributions occurring during the image integration time to        provide an EDOF image that is substantially focused throughout a        larger depth of field than the imaging system provides at a        single focal position.

It will be understood that in the description immediately above, whenthe blurred image contributions are computationally removed, the firstpreliminary image may be a blurred image that initially includes theblurred image contributions. The first preliminary image in this casecomprises detected and/or recorded image data. Processing the firstpreliminary image to remove the blurred image contributions comprisescomputational processing to the first preliminary image data, to providean EDOF image (a second or modified image) that is substantially focusedthroughout a larger depth of field than the imaging system provides at asingle focal position. Thus, the first preliminary image and theprovided EDOF image comprise different images and/or image data in thisembodiment.

In contrast, when the blurred image contributions are removed using anoptical filter and passive Fourier image-processing methods, the firstpreliminary image and the EDOF image occur simultaneously, and the firstpreliminary image need not be a detected or recorded image. Processingthe first preliminary image to remove the blurred image contributionscomprises passive optical processing to the first preliminary imagelight that is input to the EDOF imaging system, to provide an EDOF imageat the output or detector of the EDOF imaging system that issubstantially focused throughout a larger depth of field than theimaging system provides at a single focal position. Thus, it may beconsidered in such an embodiment that the first preliminary image isoptically processed during its passage through the EDOF imaging systemand prior to detection at the camera or detector of the EDOF imagingsystem, such that the provided EDOF image is the only detected orrecorded image in such an embodiment.

Control for modulation of the focus position, according to any of themethods outlined herein and/or as illustrated in FIG. 4, may beaccomplished as outlined with reference to the EDOF mode element 130 eand the lens control interface 134 shown in FIG. 2, and/or the EDOF lenscontrollers 374 and 574 shown in FIG. 3 and FIG. 5, respectively.

Because an EDOF imaging system configured according to the principlesdisclosed herein offers high speed extended depth-of-field imaging, suchan imaging system may be utilized to repetitively collect extendeddepth-of-field images at a high rate, e.g., for video imaging at 30frames per second or higher, and the plurality of extendeddepth-of-field images may be displayed as real-time video frames.

In some embodiments it is possible to make an adjustment to a controlsignal component related to the nominal center of the range R of theperiodic modulation in response to a user input (e.g., using a userinterface feature of the EDOF mode element 131 e), such that theperiodic modulation takes place about a desired nominal center of therange. In some embodiments, such an adjustment may even be controlled tovary automatically during an image exposure, to further extend a focusrange beyond that achieved by a single periodic modulation, for example.

It should be appreciated that while the timing diagram shows 7 periodsof the modulation of the focal height for each frame exposure, forpurposes of illustration, in various embodiments, a machine visioninspection system configured according to the principles disclosedherein may comprise an imaging system that modulates the focal heightover a much greater number of periods per frame exposure. For example,an exemplary imaging system may collect video images at 30 frames persecond and may modulate the focus height at a rate of 30 kHz, whichtherefore provides 1,000 periods of focus height modulation per frameexposure. One advantage of such a configuration is that the timingrelationship between the frame exposure in the periodic modulation isnot critical. For example, Equation 1 shows that an integrated pointspread function used to remove blurred image contributions depends onthe focal position as a function of time throughout an image exposure.If the assumed integrated point spread function does not match theactual focal position as a function of time throughout the imageexposure, then the blurred image contributions will not be treated in anideal manner. If the assumed integrated point spread function is basedon a full periodic modulation of the focus throughout the focus range,and only a single period (or a few periods) of a periodic focusmodulation is used during an image exposure then, if the exposure isterminated after a non-integer number of periods, the actual integratedactual focal position may be significantly “unbalanced,” in comparisonto the assumed integrated point spread function. In contrast, if theaccumulated number of periods is significant, e.g., at least 5 periodsor preferably many more, during an image exposure, then, if the exposureis terminated after a non-integer number of periods, the unbalancedcontribution of the incomplete period may be relatively insignificant,and the assumed integrated point spread function will operate in anearly ideal manner.

In some embodiments, collecting a first image during the course of atleast one period of the periodically modulated focus position maycomprise exposing an image during the course of an integer number ofperiods. Based on the foregoing discussion, it will be understood thatthis may be particularly valuable when an EDOF image exposure comprisesrelatively few periods of a periodic focus modulation (e.g., 5 or fewerperiods.) For example, this might occur when an exposure time must berelatively short in order to avoid overexposure, and/or freeze motion,or the like.

In the example shown in the timing diagram 400, the focus position ismodulated sinusoidally. In some embodiments, the image integration timeincludes a focus change over the entirety of the desired focus range(e.g., at least one period of the periodically modulated focus position,as shown in FIG. 4). In some embodiments, it may be desirable to exposean image only during the more linear portions of the sinusoidalmodulation. This allows for more balanced exposure times for each heightwithin the focus position modulation (e.g., the relatively longer focusposition dwell times at the extremes of a sinusoidal focus modulationmay be eliminated.) Thus, in some embodiments, exposing an image duringan image integration time comprises providing illumination having anintensity variation (e.g., an on/off cycle or a more gradual intensityvariation) synchronized with the periodically modulated focus position,such that it differently influences the respective exposurecontributions for different respective focus positions within the rangeof the periodically modulated focus position. It will be appreciatedthat a frame exposure may receive substantially no image contributionwhen the strobe illumination is off. The timing diagram 400 indicatestwo exemplary integration periods IPA and IPB, which may be used forexposing an image. The exemplary integration periods IPA and IPB excludethe regions near the extrema of the sinusoidal modulation, i.e., bothare at least 15 percent of the period length away from extrema portionsof the sinusoidal modulation. The integration periods IPA and IPB may becontrolled by providing a corresponding strobe illumination during aframe exposure, according to known methods.

FIG. 5 shows a schematic diagram of a second embodiment of an EDOFimaging system 500 that may be adapted to a machine vision inspectionsystem and operated according to the principles disclosed herein. Theimaging system 500 is similar to the imaging system 300 of FIG. 3.Similarly numbered elements marked 3XX in FIGS. 3 and 5XX in FIG. 5 maybe understood to be similar or identical and only significantdifferences will be described with respect to FIG. 5. The embodiment ofan EDOF imaging system shown in FIG. 5 is usable when an EDOF imagingsystem is configured to perform passive optical deconvolution ofpreliminary image light in an EDOF imaging system and output arelatively clear EDOF image to a camera and/or detector of the imagingsystem in real time. In the embodiment shown in FIG. 5, the imagingsystem 500 additionally comprises a first filtering lens 553, a secondfiltering lens 554 and an optical deconvolution filter 556. The firstfiltering lens 553 and the second filtering lens 554 provide a 4 foptical relay with the optical deconvolution filter 556 placed at aFourier plane. The optical deconvolution filter 556 may be derived froman integrated point spread function determined for the imaging system500, as described in greater detail below with reference to FIG. 6. Inoperation, the optical deconvolution filter 556 is configured to inputpreliminary image light from a variable focal length lens 570 andprocess that image light by optically filtering it, which provides anoutput EDOF image to a camera 560, which is a relatively clear EDOFimage output to the camera 560 in real time.

FIG. 6A is a graph 600A characterizing a first exemplary optical filterwhich may be used at a Fourier plane of an EDOF imaging system (e.g., asan embodiment of the optical deconvolution filter 556 of FIG. 5) inorder to perform optical deconvolution of an image from an EDOF imagingsystem and provide a relatively clear EDOF image in real time. The graphshows an optical transmission curve 610A. The optical transmission curve610A comprises a linear optical transmission profile that is, at aminimum, at the center of the optical filter. At the periphery of theoptical filter near the edge of a pupil diameter, the opticaltransmission value is at 100 percent. Beyond the pupil diameter, theoptical transmission is at zero. The optical filter characterized by thegraph 600A acts as a high pass spatial filter in the process ofdeconvolution.

FIG. 6B is a graph 600B characterizing a second exemplary optical filterthat may be used at a Fourier plane of an EDOF imaging system (e.g., asan embodiment of the optical deconvolution filter 556 of FIG. 5) inorder to perform optical deconvolution of an image from an EDOF imagingsystem and provide a relatively clear EDOF image in real time. The graphshows an optical transmission curve 610B. The optical transmission curve610B comprises a quadratic optical transmission profile that is, at aminimum, at the center of the optical filter. At the periphery of theoptical filter near the edge of a pupil diameter, the opticaltransmission value is at 100 percent. Beyond the pupil diameter, theoptical transmission is at zero. The optical filter characterized by thegraph 600B also acts as a high pass spatial filter in the process ofdeconvolution. It should be appreciated that the optical filterscharacterized by the graph 600A and the graph 600B are exemplary and notlimiting, and optical filters with other transmission profiles may beused, e.g., phase-modifying filters.

FIG. 7 is a flow diagram 700 showing one embodiment of a method foroperating an imaging system and associated signal processing of amachine vision inspection system in order to perform computationaldeconvolution of a preliminary image from an EDOF imaging system andprovide a relatively clear EDOF image approximately in real time.

At a block 710, a workpiece is placed in a field of view of the machinevision inspection system.

At a block 720, a focus position of the imaging system is periodicallymodulated over a plurality of focus positions along a focus axisdirection without macroscopically adjusting the spacing between elementsin the imaging system. The focus position is periodically modulated in afocus range including a surface height of the workpiece at a frequencyof at least 300 Hz (or a much higher frequency, in some embodiments).

At a block 730, a first preliminary image is exposed during an imageintegration time while modulating the focus position in the focus range.

At a block 740, data from the first preliminary image is processed toremove blurred image contributions occurring in the focus range duringthe image integration time to provide an image that is substantiallyfocused throughout a larger depth of field than the imaging systemprovides at a single focal position.

It may be thought that when using a very high speed periodicallymodulated variable focus lens, such as a TAG lens, that the focusposition changes so quickly that the only way it may be used to acquirean EDOF image is to continuously expose the EDOF image in the focusrange of the high speed variable focus lens, for example, as in someexamples outlined above. However this method of EDOF image exposure hascertain disadvantages in various implementations. For example, onedrawback with the method when using a periodically modulated variablefocus lens, is that the focus position changes sinusoidally, and not ata constant rate. This means a continuous (including partiallycontinuous) EDOF image exposure is not uniform throughout the focusrange, which is detrimental in a number of implementations. Analternative method of acquiring an EDOF image using such a lens, whichmay be more desirable in certain implementations, is described below.The alternative method includes using a plurality of discrete imageexposure increments to acquire a preliminary EDOF image in a focusrange, according to principles described below. Such a method may be amore adaptable, accurate, and/or robust method in variousimplementations. It should be appreciated that the focus position maychange so quickly when using a very high speed periodically modulatedvariable focus lens (e.g., a TAG lens), that significant timing,control, and “exposure amount” problems may arise in practical systems.In order to provide a practical solution to such problems, the discreteimage exposure increments that are used as constituents of an EDOF imageexposure are acquired over a plurality of periodic focus modulations,according to principles disclosed below.

FIGS. 8A-8C show exemplary timing diagrams 800A-800C, respectively,illustrating various aspects of three different image exposureimplementations suitable for an EDOF imaging system (e.g., the imagingsystem 300). The timing diagrams 800A-800C are somewhat analogous to thetiming diagram 400, in that an EDOF image exposure may be acquiredduring a periodic modulation of the focal height or focal position of avariable focus imaging system over its focus range. However, in contrastwith the implementation illustrated in the timing diagram 400, which mayuse a continuous image exposure, in the implementations shown in timingdiagrams 800A-800C, an EDOF imaging system is configured to expose apreliminary image using an image exposure comprising a plurality ofdiscrete image exposure increments, according to principles describedbelow.

In particular, the timing diagram 800A shows the periodically modulatedfocus position MFP of a variable focus imaging system, which isperiodically modulated (as shown along the time axis) over a pluralityof focus positions along a focus axis direction (as shown along thefocal plane Z position axis), over a focus range FR which is assumed toinclude the distance(s) to the surface of a workpiece to be imaged bythe variable focus imaging system. The variable focus imaging system iscapable of being operated at a very high focus modulation frequency(e.g., at least 3 kHz, or 30 kHz, or more, in various implementations).As shown in the diagram 800A, a preliminary image is exposed using animage exposure comprising a plurality of discrete image exposureincrements EI acquired at respective focus positions FP (e.g., arespective one of the focus positions Z1-Z8) during a camera imageintegration time comprising a plurality of periods of the periodicallymodulated focus position MFP. Note: The reference abbreviations EIand/or FP (and/or CT, for controlled timing, used below and shown inFIG. 9) may include an index numeral “i”, which designates a particular“ith” exposure increment EI, or focus position FP, or controlled timingCT. In the case of the exposure increment EI, the index numeral “i”generally ranges from 1 to up to the number of discrete image exposureincrements included in the preliminary image exposure (e.g., EI₁-EI₁₆,in the example illustrated in FIG. 8A.)

The plurality of discrete image exposure increments EI are eachdetermined by a respective instance of an illumination source strobeoperation or a camera shutter strobe operation, that has a respectivecontrolled timing that defines the discrete focus position FP of thecorresponding discrete image exposure increment EI. It will beappreciated that a variable focus imaging system having a periodicallymodulated focus position has a particular focus position at a particulartiming or phase within each period of the modulation. The instantaneousphase of the periodic modulation may be known based on either the drivesignal of the variable focus lens, or by monitoring the focus positiondirectly, or the like. Therefore, knowing a calibration relationshipbetween the focus position and the phase of the periodic modulation, astrobe element (e.g., strobe illumination source, or a fast electroniccamera shutter) can be controlled to briefly enable an exposure at aparticular phase timing in order to acquire an exposure increment at adesired corresponding focus position. This principle may be understoodin greater detail with reference to the U.S. Pat. Nos. 8,194,307 and9,143,674, for example, which are hereby incorporated herein byreference in their entirety. Various aspects of using a controlledtiming, as outlined above, are described in greater detail below withreference to FIG. 9.

As shown in FIG. 800A, the respective controlled timings (e.g., asrepresented by their respective increment times T1-T16) are distributedover a plurality of periods of the periodically modulated focus positionMFP, and are configured to provide a set of discrete focus positions FP,which are approximately evenly spaced along the focus axis direction (asrepresented by their focus position values Z1-Z8.) It has beendetermined that evenly spaced and/or “weighted” exposure contributionsto a “raw” or preliminary EDOF image may be advantageous in terms ofsignal processing and/or computational operations that may besubsequently performed to enhance the EDOF image. For example, suchexposure contributions may be advantageous when providing an enhancedEDOF image by performing deconvolution operations on the preliminaryEDOF image, by using a blur kernel that characterizes variable focusimaging system throughout its focus range. Such deconvolution operationsare described, for example, in Publication No. WO2009120718 A1, which ishereby incorporated herein by reference in its entirety. However, knownmethods of providing evenly spaced and/or weighted exposurecontributions throughout a focus range for an EDOF image are notsufficiently fast, accurate (in terms of EDOF image clarity andquality), or repeatable.

As previously outlined, the focus position may change so quickly whenusing a very high speed periodically modulated variable focus lens(e.g., a TAG lens), that significant timing, control, and “exposureamount” problems may arise in practical systems. In particular, duringany particular modulation the imaging system focus position maysequentially pass through adjacent pairs of desired focus positionswithin a period of tens of nanoseconds, rendering discrete exposures atsuch sequential adjacent focus positions, impractical and/or inaccurate.In order to provide a practical solution to such problems, therespective controlled timings used to acquire the discrete imageexposure increments EI at the desired evenly spaced focus positions FP(e.g., Z1-Z8) are configured so that for a plurality of adjacent pairsof discrete focus positions in the set (e.g., the set Z1-Z8), when afirst controlled timing provides a first discrete focus position set ofthe adjacent pair, a second controlled timing that provides a seconddiscrete focus position of the adjacent pair is controlled to have adelay relative to the first controlled timing such that the secondcontrolled timing is controlled to occur after N reversals of thedirection of the focus position change during its periodic modulationfollowing the first controlled timing, where N is at least 1. Suchdirection reversals occur at the limits of the focus range FR, that is,at the extrema of the sinusoidally modulated focus position MFP. Invarious implementations, this principle is followed for all of theadjacent pairs of discrete focus positions in the set. Such a timingconfiguration makes it practical to acquire discrete image exposureincrements at closely spaced focus positions, in a practical,economical, and versatile manner, with good accuracy.

To clarify the operation with respect to diagram 800A, during theperiodically modulated focus position MFP, the camera image integrationtime starts in the discrete image exposure increment EI₁ is acquired atthe increment time T1, which has a phase timing tz1 that corresponds tothe desired focus position FP=Z1. The periodically modulated focusposition MFP then continues through adjacent focus position Z2, andpositions Z3 and Z4, in this example, wherein it is assumed that it isnot practical to complete the operations necessary to obtain the nextdiscrete image exposure increment before the focus position reaches Z4.In contrast, the next practical time (that is, after sufficient elapsedtime) to obtain a discrete image exposure increment is at the time T2,when the focus position reaches Z5. The discrete image exposureincrement EI2 is acquired at the increment time T2, which has a phasetiming tz5 that corresponds to the desired focus position FP=Z5. Thenext practical time (that is, after sufficient elapsed time) to obtain adiscrete image exposure increment is at the time T3. The discrete imageexposure increment EI3 is acquired at the increment time T3, which has aphase timing tz8 that corresponds to the desired focus position FP=Z8.The acquisition of discrete image exposure increments EI continues in ananalogous fashion through the acquisition of the discrete image exposureincrement EI8, acquired at the increment time T8, which has a phasetiming tz4 that corresponds to the desired focus position FP=Z4. Up tothis point, discrete image exposure increments EI₁-EI₈ have beenacquired at each of the desired evenly spaced focus positions Z1-Z8(designated as a subset 810A′), during the image integration time. Insome implementations, the image integration time could be terminated atthis point. However in the example illustrated in the diagram 800A, thebrightness and/or “image signal” of the preliminary EDOF image exposureis strengthened by repeating the previous acquisition pattern to obtainthe discrete image exposure increments EI₉-EI₁₆ at the times T9-T16,corresponding to each of the desired evenly spaced focus positions Z1-Z8(designated as a subset 810A″). The image integration time is thenterminated. It will be appreciated that this maintains even “imageweighting” for each of the focus positions Z1-Z8 in the overallpreliminary EDOF image exposure, which comprises the set 810A ofdiscrete image exposure increments and/or evenly spaced focus positions.

The first preliminary EDOF image (e.g., its image data, as provided by adigital camera or the like), exposed as outlined above, may be processedto remove blurred image contributions occurring in the focus rangeduring the image integration time to provide an extended depth of field(EDOF) image that is substantially focused throughout a larger depth offield than the imaging system provides at a single focal position. Forexample, in one implementation, processing the preliminary EDOF image toremove blurred image contributions may comprises deconvolutionprocessing of its image data using a predetermined function thatcharacterizes the imaging system (e.g., an integrated point spreadfunction that characterizes the imaging system over a focus rangecorresponding to the evenly spaced focus positions), to provide aclearer EDOF image.

In the diagram 800A, each discrete image exposure increment EI isacquired when the focus position is changing in the same direction. Insome implementations more accurately spaced and/or repeatable focuspositions are provided (in comparison to acquiring exposure incrementsduring both directions of focus position change, as described below withreference to FIG. 8C). It may be noted that in this case, when a firstcontrolled timing provides a first discrete focus position set of anadjacent pair (e.g., EI₁ at Z1), a second controlled timing thatprovides a second discrete focus position of the adjacent pair (e.g.,EI₄ at Z2) is controlled to have a delay relative to the firstcontrolled timing such that the second controlled timing is controlledto occur after N reversals of the direction of the focus position, whereN is at least 2.

The timing diagram 800B is similar to the timing diagram 800A, and maygenerally be understood by analogy, except as otherwise indicated below.In the diagram 800B, the plurality of discrete image exposure incrementsEI₁-EI₈ are each determined by a respective instance of an illuminationsource strobe operation or a camera shutter strobe operation that has arespective controlled timing that defines its discrete focus position FP(e.g., one of the evenly spaced focus positions Z1-Z8.) The respectivecontrolled timings (e.g., as represented by their respective incrementtimes T1-T8) are distributed over a plurality of periods of theperiodically modulated focus position MFP, within the image integrationtime.

To clarify the operation with respect to diagram 800B, during theperiodically modulated focus position MFP, the camera image integrationtime starts in the discrete image exposure increment EI₁ that isacquired at the increment time T1, which has a phase timing tz1 thatcorresponds to the desired focus position FP=Z1. The periodicallymodulated focus position MFP then continues through two reversals of thedirection of the focus position change during its periodic modulationfollowing the increment time T1. Even when the periodic modulation has avery high frequency, it is then practical to obtain a discrete imageexposure increment EI₂ at the time T2, which has a phase timing tz2 thatcorresponds to the desired focus position FP=Z2, which is adjacent toZ1. The acquisition of discrete image exposure increments EI continuesin an analogous fashion through the acquisition of the discrete imageexposure increment EI8, acquired at the increment time T8, which has aphase timing tz8 that corresponds to the desired focus position FP=Z8.At this point, discrete image exposure increments EI₁-EI₈ have beenacquired at each of the desired evenly spaced focus positions Z1-Z8(forming a 810B), during the image integration time. In this example,the image integration time is terminated at this point. It will beappreciated that this maintains even “image weighting” for each of thefocus positions Z1-Z8 in the overall preliminary EDOF image exposure,which comprises the set 810B of discrete image exposure incrementsand/or evenly spaced focus positions.

The timing diagram 800C is similar to the timing diagram 800A, and maygenerally be understood by analogy, except as otherwise indicated below.In the diagram 800C, the plurality of discrete image exposure incrementsEI₁-EI₁₆ are each determined by a respective instance of an illuminationsource strobe operation or a camera shutter strobe operation that has arespective controlled timing that defines its discrete focus position FP(e.g., one of the evenly spaced focus positions Z1-Z8.) The respectivecontrolled timings (e.g., as represented by their respective incrementtimes T1(=tz1), T2(=tz4), T3(=tz8), T4(=tz7), and so on, are distributedover a plurality of periods of the periodically modulated focus positionMFP, within the image integration time.

To clarify the operation with respect to diagram 800C, the discreteimage exposure increment EI₁ is acquired at the increment time T1, whichhas a phase timing tz1 that corresponds to the desired focus positionFP=Z1. The periodically modulated focus position MFP then continuesthrough adjacent focus position Z2, and position Z3 in this example,wherein it is assumed that it is not practical to complete theoperations necessary to obtain the next discrete image exposureincrement before the focus position reaches Z3. In contrast, the nextpractical time (that is, after sufficient elapsed time) to obtain adiscrete image exposure increment is at the time T2, when the focusposition reaches Z4. The discrete image exposure increment EI2 isacquired at the increment time T2, which has a phase timing tz4 thatcorresponds to the desired focus position FP=Z4. The next practical time(that is, after sufficient elapsed time) to obtain a discrete imageexposure increment is at the time T3. The discrete image exposureincrement EI3 is acquired at the increment time T3, which has a phasetiming tz8 that corresponds to the desired focus position FP=Z8. Thefocus position change slows as it reverses direction after the timeT3(=tz8), such that the next practical time to obtain a discrete imageexposure increment is at the time T4 which has a phase timing tz7 thatcorresponds to the desired focus position FP=Z7. It may be noted thatthis focus position is adjacent to the focus position Z8 of thepreviously acquired discrete image exposure increment, after just onereversal (N=1) of the direction of focus change. It may be noted thatN=1, because in this example discrete image exposure increments areacquired during both directions of focus change. More generally, it maybe noted that in this example when a first controlled timing provides afirst discrete focus position set of an adjacent pair, a secondcontrolled timing that provides a second discrete focus position of theadjacent pair may occur after various numbers of reversals of thedirection of the focus position (N ranges from 1 to 4, for variousadjacent pairs, in this example.)

In the example illustrated in the diagram 800C, the brightness and/or“image signal” of the preliminary EDOF image exposure is strengthened byrepeating the discrete image exposure increments at each focus position(but in a different order the second time). The discrete image exposureincrements EI₁-EI₈ obtained at the times T1-T8 (designated in a subset810C′ in FIG. 8C), correspond to each of the desired evenly spaced focuspositions Z1-Z8. The discrete image exposure increments EI₉-EI₁₆ at thetimes T9-T16 (designated in a subset 810C″ in FIG. 8C) provide arepeated discrete image exposure increment corresponding to each of thedesired evenly spaced focus positions Z1-Z8. Together, the subsets 810C′and 810C″ contribute to the overall preliminary EDOF image exposure,which comprises the set 810C of discrete image exposure incrementsand/or evenly spaced focus positions. The pattern or repetitionillustrated here (and in the diagram 800A) is not limiting. Moregenerally, in various embodiments, repeating the discrete image exposureincrements at each focus position may be configured wherein a pluralityof discrete image exposure increments used for a preliminary EDOF imagecomprise at least a first instance and a second instance of a discreteimage exposure increment acquired at each discrete focus position (e.g.,each of at least 20 approximately equally spaced focus positions, insome implementations), during the image integration time. The respectivecontrolled timings used for acquiring the first and second instances ofa discrete image exposure increment at the same discrete focus positionmay be configured such that a controlled timing used for the secondinstance has a delay relative to a controlled timing used for the firstinstance, and is controlled to occur after M reversals of the directionof change of the focus position during its periodic modulation followingthe controlled timing used for the first instance, where M is at least1.

It will be appreciated that the foregoing timing diagram examples areexemplary only and not limiting. Other timing configurations andcombinations may be realized based on principles illustrated anddescribed above. In some implementations, the focus range may span atleast 10 times the depth of field than the imaging system in a singlefocal position and the respective controlled timings are configured toprovide a set of at least 20 approximately equally spaced discrete focuspositions (e.g. Z1-Z20) during the image integration time. In someimplementations, the at least 20 discrete focus positions maydistributed over at least 50% of the focus range, to provide arelatively large extended depth field. In some implementations, the atleast 20 discrete focus positions may distributed over at least 70% oreven 80% of the focus range.

In some implementations, many more discrete image exposure incrementsmay be provided during a single period of the periodically modulatedfocus position. However, in other implementations, and particularlythose with high frequency periodic modulations of the focus position, atmost 6 discrete image exposure increments might be provided during asingle period of the periodically modulated focus position.

In some implementations, the operations outlined above with respect toany of diagrams 800A-800C, or a combination thereof, may be to provide aplurality of EDOF images that are substantially focused throughout alarger depth of field than the imaging system provides at a single focalposition, and the plurality of EDOF images may be displayed in a livevideo display window that is provided on a display included in a machinevision inspection system.

In various implementations, it is currently more practical to providediscrete image exposure increments using an illumination source strobeoperation that has a respective controlled timing that defines thediscrete focus position of its corresponding discrete image exposureincrement. However, digital cameras having an electronic “shutterstrobe” function that can create timed sub-exposure increments within anoverall image integration period are increasingly available. Suchcameras may provide the controlled timings outlined above, usingcontinuous or ambient illumination, in some implementations.

In some implementations, an illumination source strobe operation may beused in combination with an illumination source that includes multiplecolor sources. In such a case, axial chromatic aberration in the imagingsystem may cause the various color sources to focus at different focuspositions. In such a case it should be appreciated that a respectivecontrolled timing outlined above may comprise a different color sourcetiming for each color source, including a timing offset between thecolor source timings that compensates for axial chromatic aberration inthe imaging system, so that each of the color sources provides the samediscrete focus position.

It will be understood that the operations described above with referenceto the timing diagrams 800A-800C may be implemented in a correspondinglyconfigured EDOF imaging system which is similar to one of the imagingsystems depicted in any of the FIG. 2, 3, or 5, for example, to providean image of a workpiece that has a larger depth of field than theimaging system provides at a single focal position.

FIG. 9 shows a timing diagram 900 which shows certain details of oneexemplary implementation of a controlled timing CT that may be used todefine a discrete focus position FP and certain other characteristics todetermine a corresponding discrete image exposure increment EI. Inparticular, the control timing may be implemented in an illuminationsource strobe operation or camera shutter strobe operation to determinethe focus position FP and certain other characteristics of acorresponding discrete image exposure increment EI. The timing diagram900 may be understood as a more detailed view of a portion of the timingdiagram 800A, showing two representative discrete image exposureincrements EI₁ and EI₂, and may generally be understood by analogythereto. However, additional principles related to one exemplaryimplementation of a controlled timing CT are described.

In the implementation shown FIG. 9, each controlled timing CTi comprisesa respective increment time Ti and a respective increment duration Di,and a respective increment illumination intensity Li is used during therespective increment duration Di. In particular, the illustratedcontrolled timing CT1 that determines the exposure increment EI₁comprises an increment time T1 (as previously described with referenceto FIGS. 8A-8C) and a respective increment duration D1 (e.g., a timedstrobe duration). The illustrated controlled timing CT2 that determinesthe exposure increment EI₂ similarly comprises a respective incrementtime T2 and an increment duration D2. It may be seen that each incrementduration is located to provide a central or average increment time thatcorresponds to a desired focus position. For example, the incrementduration D1 is located to provide the increment time T1(=tz1)corresponding to the desired focus position FP₁(=Z1), and the incrementduration D2 is located to provide the increment time T2(=tz5)corresponding to the desired focus position FP₂(=Z5). In variousimplementations, a respective increment illumination intensity Li isused during a respective increment duration Di, and each discrete imageexposure increment is exposed using a combination of its respectiveincrement illumination intensity Li and its respective incrementduration Di such that the product (Li*Di) is approximately the same foreach of the discrete image exposure increments. Although this aspect ofthe implementation is not strictly required, it tends to provide equal“weighting” in a preliminary EDOF image, at each of the desired focuspositions, which may be advantageous in some implementations.

The implementation shown in FIG. 9 also includes an aspect wherein adiscrete image exposure increment (EI₂) corresponding to a focusposition (FP2) that is relatively closer to the middle of the focusrange FR, comprises a combination of in increment duration D2 which isrelatively shorter and an increment illumination intensity (e.g., L2,not shown) which is relatively larger, and a discrete image exposureincrement (EI₁) corresponding to a focus position (FP1) that isrelatively farther from the middle of the focus range FR comprises acombination of a second increment duration D1 which is relativelylonger, and a second increment illumination intensity (e.g., L1, notshown) which is relatively smaller. In various implementations whereinthe periodically modulated focus position changes approximatelysinusoidally as a function of time, this allows the product (Li*Di) tobe approximately the same for each of the discrete image exposureincrements while at the same time allowing each respective incrementduration to be controlled to provide approximately the same amount offocus position change ΔFP during each increment duration. For example,it may be seen that this allows ΔFP1=ΔFP2 in FIG. 9, even though therate of focus change is different for each exposure increment due to thesinusoidal focus modulation. Although this aspect of the implementationis also not strictly required, it tends to provide another aspect ofequal “weighting” in a preliminary EDOF image, at each of the desiredfocus positions, which may be advantageous in some implementations.

FIG. 10 is a flow diagram 1000 showing one embodiment of a method foroperating an imaging system of a machine vision inspection system inorder to provide at least one EDOF image that has a larger depth offield than the imaging system in a single focal position. The methodincludes exposing a preliminary EDOF image using an image exposurecomprising a plurality of discrete image exposure increments, accordingto principles disclosed herein.

At a block 1010, a workpiece is placed in a field of view of the machinevision inspection system.

At a block 1020, a focus position of the imaging system is periodicallymodulated without macroscopically adjusting the spacing between elementsin the imaging system, wherein the focus position is periodicallymodulated over a plurality of focus positions along a focus axisdirection in a focus range including a surface height of the workpiece,at a modulation frequency of at least 3 kHz.

At a block 1030, a first preliminary image is exposed using an imageexposure comprising a plurality of discrete image exposure incrementsacquired at respective discrete focus positions during an imageintegration time comprising a plurality of periods of the periodicallymodulated focus position, wherein:

-   -   the plurality of discrete image exposure increments are each        determined by a respective instance of an illumination source        strobe operation, or a camera shutter strobe operation, that has        a respective controlled timing that defines the discrete focus        position of the corresponding discrete image exposure increment;    -   the respective controlled timings are distributed over the        plurality of periods of the periodically modulated focus        position, and are configured to provide a set of discrete focus        positions which are approximately evenly spaced along the focus        axis direction; and    -   the respective controlled timings are furthermore configured so        that for a plurality of adjacent pairs of discrete focus        positions in the set, when a first controlled timing provides a        first discrete focus position set of the adjacent pair, a second        controlled timing that provides a second discrete focus position        of the adjacent pair is controlled to have a delay relative to        the first controlled timing such that the second controlled        timing is controlled to occur after N reversals of the direction        of change of the focus position during its periodic modulation        following the first controlled timing, where N is at least 1.

At a block 1040, the first preliminary image is processed to removeblurred image contributions occurring in the focus range during theimage integration time to provide an extended depth of field (EDOF)image that is substantially focused throughout a larger depth of fieldthan the imaging system provides at a single focal position. Forexample, such processing may include performing deconvolution operationsusing a blur kernel that characterizes the imaging system throughout itsfocus range (e.g., and integrated point spread function), as previouslydiscussed herein.

While various embodiments of the invention have been illustrated anddescribed, numerous variations in the illustrated and describedarrangements of features and sequences of operations will be apparent toone skilled in the art based on this disclosure. Thus, it will beappreciated that various changes can be made therein without departingfrom the spirit and scope of the invention.

The embodiments of the invention in which an exclusive property orprivilege is claimed are defined as follows:
 1. A method for operatingan imaging system of a machine vision inspection system to provide atleast one image that has a larger depth of field than the imaging systemin a single focal position, the method comprising: (a) placing aworkpiece in a field of view of the machine vision inspection system;(b) periodically modulating a focus position of the imaging systemwithout macroscopically adjusting the spacing between elements in theimaging system, wherein the focus position is periodically modulatedover a plurality of focus positions along a focus axis direction in afocus range including a surface height of the workpiece, at a modulationfrequency of at least 3 kHz; (c) exposing a first preliminary imageusing an image exposure comprising a plurality of discrete imageexposure increments acquired at respective discrete focus positionsduring an image integration time comprising a plurality of periods ofthe periodically modulated focus position, wherein: the plurality ofdiscrete image exposure increments are each determined by a respectiveinstance of an illumination source strobe operation, or a camera shutterstrobe operation, that has a respective controlled timing that definesthe discrete focus position of the corresponding discrete image exposureincrement; the respective controlled timings are distributed over theplurality of periods of the periodically modulated focus position, andare configured to provide a set of discrete focus positions which areapproximately evenly spaced along the focus axis direction; and therespective controlled timings are furthermore configured so that for aplurality of adjacent pairs of discrete focus positions in the set, whena first controlled timing provides a first discrete focus position setof the adjacent pair, a second controlled timing that provides a seconddiscrete focus position of the adjacent pair is controlled to have adelay relative to the first controlled timing such that the secondcontrolled timing is controlled to occur after N reversals of thedirection of change of the focus position during its periodic modulationfollowing the first controlled timing, where N is at least 1; and (d)processing the first preliminary image to remove blurred imagecontributions occurring in the focus range during the image integrationtime to provide an extended depth of field (EDOF) image that issubstantially focused throughout a larger depth of field than theimaging system provides at a single focal position.
 2. The method ofclaim 1, wherein in step (c) each discrete image exposure increment isdetermined by a respective instance of the illumination source strobeoperation.
 3. The method of claim 1, wherein the periodically modulatedfocus position changes approximately sinusoidally as a function of time.4. The method of claim 1, wherein each “ith” respective controlledtiming comprises a respective increment time Ti and a respectiveincrement duration Di, and a respective increment illumination intensityLi is used during the respective increment duration Di, and eachdiscrete image exposure increment of the first preliminary image isexposed using a combination of its respective increment illuminationintensity Li and its respective increment duration Di such that theproduct (Li*Di) is approximately the same for each of the discrete imageexposure increments.
 5. The method of claim 4, wherein a discrete imageexposure increment corresponding to a first focus position that isrelatively closer to the middle of the focus range comprises acombination of a first increment duration Di which is relatively shorterand a first increment illumination intensity Li which is relativelylarger, and a discrete image exposure increment corresponding to asecond focus position that is relatively farther from the middle of thefocus range comprises a combination of a second increment duration Diwhich is relatively longer and a second increment illumination intensityLi which is relatively smaller.
 6. The method of claim 5, wherein theperiodically modulated focus position changes approximately sinusoidallyas a function of time and the first increment duration and the secondincrement duration are controlled to provide approximately the sameamount of focus position change during the first increment duration andthe second increment duration.
 7. The method of claim 1, wherein thefocus range spans at least 10 times the depth of field than the imagingsystem in a single focal position and the respective controlled timingsare configured to provide a set of at least 20 discrete focus positionsduring the image integration time.
 8. The method of claim 7, wherein:the imaging system comprises a tunable acoustic gradient index ofrefraction (TAG) lens configured to periodically modulate the focusposition of the imaging system without macroscopically adjusting thespacing between elements in the imaging system; the modulation frequencyis at least 30 kHz; and at most 6 discrete image exposure increments areprovided during a single period of the periodically modulated focusposition.
 9. The method of claim 7, wherein the at least 20 discretefocus positions are distributed over at least 50% of the focus range.10. The method of claim 7, wherein: the first plurality of discreteimage exposure increments used for the first preliminary image compriseat least a first instance and a second instance of a discrete imageexposure increment acquired at each of the at least 20 discrete focuspositions, during the image integration time, and the respectivecontrolled timings used for acquiring the first and second instances ofa discrete image exposure increment at the same discrete focus positionare configured such that a controlled timing used for the secondinstance has a delay relative to a controlled timing used for the firstinstance, and is controlled to occur after M reversals of the directionof change of the focus position during its periodic modulation followingthe controlled timing used for the first instance, where M is atleast
 1. 11. The method of claim 1, wherein processing the firstpreliminary image to remove blurred image contributions comprisesdeconvolution processing of image data corresponding to the firstpreliminary image using a predetermined function that characterizes theimaging system, to provide the EDOF image.
 12. The method of claim 1,wherein: the imaging system comprises a variable focal length lens, andperiodically modulating a focus position of the imaging system comprisesmodulating a focus position of the variable focal length lens, and theimaging system further comprises an optical filter located to receiveand spatially filter preliminary image light from the variable focallength lens; and in step (d), processing the first preliminary image toremove blurred image contributions comprises spatially filtering thepreliminary image light using the optical filter, to provide the EDOFimage based on light output by the optical filter.
 13. The method ofclaim 1, further comprising: repeating steps (c) and (d) to provide aplurality of EDOF images that are substantially focused throughout alarger depth of field than the imaging system provides at a single focalposition, and displaying the plurality of EDOF images of the workpiecein a live video display window that is provided on a display included inthe machine vision inspection system.
 14. The method of claim 1, whereinthe illumination source includes multiple color sources, and therespective controlled timing comprises a different color source timingfor each color sources, including a timing offset between the colorsource timings that compensates for axial chromatic aberration in theimaging system so that each of the color sources provides the samediscrete focus position.
 15. An imaging system for providing at leastone image of a workpiece that has a larger depth of field than theimaging system provides at a single focal position, the imaging systemcomprising: an objective lens, a variable focal length tunable acousticgradient index of refraction (TAG) lens, and a camera, at least one of acontrollable strobe illumination light source or a fast camera shutteroperable within the camera during an image integration time; and acontrol system configured to control the camera, and the strobeillumination light source if present, and to control the TAG lens toperiodically modulate the focus position of the imaging system withoutmacroscopically adjusting the spacing between elements in the imagingsystem, wherein the control system is further configured to: (a) controlthe TAG lens to periodically modulate the focus position over aplurality of focus positions along a focus axis direction in a focusrange including a surface height of the workpiece, at a modulationfrequency of at least 30 kHz; (b) operate the imaging system to expose afirst preliminary image using an image exposure comprising a pluralityof discrete image exposure increments acquired at respective discretefocus positions during an image integration time comprising a pluralityof periods of the periodically modulated focus position, wherein: theplurality of discrete image exposure increments are each determined by arespective instance of an illumination source strobe operation, or acamera shutter strobe operation, that has a respective controlled timingthat defines the discrete focus position of the corresponding discreteimage exposure increment, the respective controlled timings aredistributed over the plurality of periods of the periodically modulatedfocus position, and are configured to provide a set of discrete focuspositions which are approximately evenly spaced along the focus axisdirection, and the respective controlled timings are furthermoreconfigured so that for a plurality of adjacent pairs of discrete focuspositions in the set, when a first controlled timing provides a firstdiscrete focus position set of the adjacent pair, a second controlledtiming that provides a second discrete focus position of the adjacentpair is controlled to have a delay relative to the first controlledtiming such that the second controlled timing is controlled to occurafter N reversals of the direction of change of the focus positionduring its periodic modulation following the first controlled timing,where N is at least 1; and (c) process the first preliminary image toremove blurred image contributions occurring in the focus range duringthe image integration time to provide an extended depth of field (EDOF)image that is substantially focused throughout a larger depth of fieldthan the imaging system provides at a single focal position.
 16. Theimaging system of claim 15, wherein each “ith” respective controlledtiming comprises a respective increment time Ti and a respectiveincrement duration Di, and a respective increment illumination intensityLi is used during the respective increment duration Di, and eachdiscrete image exposure increment of the first preliminary image isexposed using a combination of its respective increment illuminationintensity Li and its respective increment duration Di such that theproduct (Li*Di) is approximately the same for each of the discrete imageexposure increments.
 17. The imaging system of claim 16, wherein: adiscrete image exposure increment corresponding to a first focusposition that relatively closer to the middle of the focus rangecomprises a combination of a first increment duration Di which isrelatively shorter and a first increment illumination intensity Li whichis relatively larger, and a discrete image exposure incrementcorresponding to a second focus position that is relatively farther fromthe middle of the focus range comprises a combination of a secondincrement duration Di which is relatively longer and a second incrementillumination intensity Li which is relatively smaller; the periodicallymodulated focus position changes approximately sinusoidally as afunction of time; and the first increment duration and the secondincrement duration are controlled to provide approximately the sameamount of focus position change during the first increment duration andthe second increment duration.
 18. The imaging system of claim 15,wherein the focus range spans at least 10 times the depth of field ofthe imaging system in a single focal position and the respectivecontrolled timings are configured to provide a set of at least 20discrete focus positions during the image integration time.
 19. Theimaging system of claim 15, wherein at most 6 discrete image exposureincrements are provided during a single period of the periodicallymodulated focus position.
 20. The imaging system of claim 15, whereinprocessing the first preliminary image to remove blurred imagecontributions comprises deconvolution processing of image datacorresponding to the first preliminary image using a predeterminedfunction that characterizes the imaging system, to provide the EDOFimage.